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ABSTRACT 

This report focuses on students who are underrepresented in 
the high academic ' achievement pool by calling attention to the important role 
that relational data management systems can play in educational decision 
making, particularly with regard to how they can be applied to closing the 
academic achievement gap. After explaining the need for data collection and 
evaluation, the report examines elements of a data gathering system, then 
presents examples of data management systems, looking at programs and data 
support resources. The report concludes that the development of relational 
program and student data management systems provides schools with increased 
capacity to document and understand the characteristics of respective inputs 
and outcomes by categories of students, staff, and programs in relation to 
differing educational contexts and circumstances. Effective data management 
systems can make this capacity readily available in a "user friendly" format 
to people at various levels in decision making. Two sidebars offer data 
support resources and data disaggregation tools. (Contains 15 references.) 
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Toward a Relatioeal Data Maeagemeet System for Educatioe 



The Need for Data Collection and 
Evaluation 

Several ideas have been advanced recently concerning ways 
to close the academic achievement gap between many students 
of color £ind their European American and Asian American coun- 
terparts. For example, previous issues of this newsletter have 
advocated for: (1) the use of knowledge generated from the scien- 
tific community regarding how people learn (Bridglall, 2001); (2) 
attention to the deliberate development of academic ability, par- 
ticularly for underachieving students of color and generally for 
all students who come from underresourced families and schools 
(Gordon, 2001); and (3) the use of supplementary education to 
promote high levels of academic and personal development 
(Bridglall & Gordon, 2002). 

In this issue of Pedagogical Inquiry and Praxis we continue 
the focus on students who are underrepresented in the high acade- 
mic achievement pool by calling attention to the important role 
that relational data management systems can play in education 
decision making, especially with regard to how they can be appHed 
to closing the academic achievement gap. We refer to decisions 
concerning education poHcy, the conduct and management of 
teaching and leeurung, and the provision of continuing profession- 
al development. Our focus on these topics is not happenstance. 
Darling-Hammond (1998) has emphasized the critical relationship 
of the level and quahty of teacher preparation and actual teaching 
practices to the academic development of students. Gordon (1986), 
Greeno (2001), and Resnick (1999) have examined the explicit 
value of learner behaviors — such as time-on-task, deployment of 
relevant effort, and resource utiHzation — ^to the quality of stu- 
dents’ academic achievement. Stdl other researchers suggest that 
governance, leadership, and organizational structure of classrooms 
and schools are the more significant determinants of educational 
effectiveness. We can agree that these differing perspectives may 
vary in their actual influence on what occurs in school. What con- 
tinues to frustrate researchers and practitioners, however, is the 
question of why certain groups of children do not perform well 
despite school interventions designed specifically to close the acad- 
emic achievement gap. 

The problem in identifying specific factors and patterns that 
influence achievement may be related to the fact that the inter- 
actions of the variables grounded in these several perspectives 
affect not only the impact of each, but also the way that the vari- 
ables interact from one situation to the other. It is thus impor- 
tant that those responsible for teaching and learning activities 
not only understand these dynamic interactions but also use that 
understanding to inform decisions regarding policy and practice 
to improve pedagogy and learning. 




Particular kinds of decision making must be informed by 
relevant information on students, staff, program, and the deploy- 
ment of resources. Fortunately, new computer technology 
increases educators’ capacities to use data systematically to 
manipulate the interactions inherent in teaching euid learning 
situations. Data systems currently exist in most education insti- 
tutions, and especially in public education systems, but the 
knowledge to use data to improve teaching and learning on 
macro levels (i.e., district, program, and school), and on micro 
levels (including classrooms euid individual learner situations), is 
lacking. Thus, we need to identify the variables that influence 
teaching and learning and develop efficient measurement and 
data gathering systems that incorporate them. In addition, we 
need to invest in capacity building — training administrators, 
supervisors, and teachers-— so that personnel can use the data to 
inform daily decisions. 

Elements of a Data Gathering System 

According to the Center for Research on Evaluation 
Standards and Student Testing (CRESST), there are three build- 
ing blocks of accountability (in addition to several key capacities) 
that support the development and implementation of a compre- 
hensive and effective accountability and data management sys- 
tem: (1) a set of policies and procedures that encourages and 
supports effective teaching and learning; (2) differential strate- 
gies for regularly eliciting and managing information that not 
only indicate how schools are functioning for all students but 
also identify specific areas of the school context that may influ- 
ence the school’s success or failure; and (3) mecheuiisms for 
reconsidering school practices when students are not well served. 

Information appropriately managed through technology 
enables the tailoring of data euialysis and management in ways 
implicit in pedagogical approaches which were first proposed as 
early as the mid twentieth century. Tyier (1949) advocated the 
use of data from tests and other assessments for data-based deci- 
sion making in the classroom for instructional purposes. Fleuia- 
gan (1967) suggested that one of the ways to address the special 
£uid individual needs of students, and of specific groups of learn- 
ers, is to customize the pedagogy. Moreover, Flanagan and 
Glaser (1967), independently working on related models, assert- 
ed that individualized and customized teaching £uid learning 
would rest on the teacher’s capacity to meuiage large amounts of 
student and instructional data and materials as the basis for 
decisions concerning the design and management of teaching 
and learning transactions. These ideas were never fully imple- 
mented, partially because educators did not have the capacity to 
manage the information necessary to design, deliver, and evalu- 






ate individualized instruction. 

We are at a juncture where we recognize that it is not only in 
the design and management of teaching and learning transactions 
that precise information is needed. Indeed, administrators and 
policy-makers need to make decisions that require their under- 
standing of education data and information on the functioning of 
the various personnel, students, programs and systems for which 
they are responsible. Yet only few school districts have the capaci- 
ty to manage these data efficiently and in ways that can inform 
critical decisions and, equally important, anticipate both intended 
and unintended consequences. Existing databases in school, dis- 
trict, and state systems are generally uncoordinated, redundant, 
and/or have large gaps in essential information. Current systems 
of data management are built in response to top-down, politically 
driven, often legislative actions, with little cognizance of the rela- 
tional information needs of the people who make day-to-day edu- 
cation policy and practice decisions. Guidance counselors, instruc- 
tional supervisors, and teachers who could use these data are 
often overwhelmed by the tasks of searching out the relevant 
information to meet their decision-making needs. 

The capacity to analyze program, staff, and student data to 
better inform education decisions now exists. In the U.S., we 
have the technology to document the dynamic interactions 
between multiple variables in diverse teaching and learning situ- 
ations. We can quickly examine relationships among a wide 
range of dependent and independent variables to determine the 
interaction of demographic characteristics, learning behaviors, 
class size, instructional behavior, teacher characteristics, curricu- 
lum demands, time on task, attendance patterns, resource avail- 
ability, and resource utilization, for example. Through the use of 
relational program and student data management systems, it is 
possible to identify program, staff, and student variables that 
are associated with particular pedagogical outcomes. 

Broadly speaking, the information products of these data 
management systems are limited only by the character and qual- 



ity of a school system’s data inputs and the particular questions 
asked of the system. Given the imperative to uncouple achieve- 
ment from such social divisions as class, ethnicity, gender, and 
first language (Coleman, 1965), and, more recently, the goals of 
the No Child Left Behind legislation, one of the critically impor- 
tant advantages of a data management system is its capacity to 
disaggregate data by specified input and outcome variables and 
to identify relationships between them. In addition, Lezotte and 
Jacoby (1992) suggest that a primary purpose of these analytic 
processes is to provide individual schools and districts with the 
mechanism for gauging their own effectiveness. 

lb that end, specific procedures can be used to identify the 
different subsets of students who master (or fail to master) the 
curriculum, by school, grade level, program, and/or course. An 
analysis of achievement by grade level, for example, can enable 
principals and district supervisors to monitor whether: (1) stu- 
dents of differing socioeconomic levels, races, and gender are 
mastering the essential student outcomes; and (2) whether the 
curriculum is equitably available, taught to and mastered by all 
students, especially those from disadvantaged and underrepre- 
sented groups. The disaggregation of data which the federal gov- 
ernment proposes to use in monitoring and monetarily reward- 
ing or punishing states whose schools do not demonstrate 
improvements on standardized tests (James, Jurich, & Estes, 
2001) is also a practical, hands-on process that enables a school 
to answer two critical questions: For whom is its curriculum 
effective? and What does the curriculum teach? 

Examples of Data Management Systems 

Mary Ann Lachat played a pivotal role in initiating one of 
the more comprehensive relational data management systems 
we have reviewed. The SOCRATES™ system (developed by 
Lachat and the Center for Resource Management [CRM]) was 
designed to enable the management and disaggregation of data, 
to track the programs and practices to which students have been 



Data Support Resources 



Toofbelt, by the North Central Regional Educational Laboratory: Includes a wealth of resources to support data-driven decision making including: 
data retreat facilitation guide, data tools, planrdng tools, surveys, check lists, etc. 
http:/ /www.ncrel.org/toolbelt/ 

Data-Driven Implementation Tools: Includes three free tools for K-12 schools to use: School Climate Inventory (SCI), Student 
Achievement Data, and School Observation Measure, 
http:/ / www.serve.org/csrd/datadriven/data.html 

Evaluating Whole-School Reform Efforts: A Guide for District and School Staff Provides an overview of conducting impact and 

implementation evaluations. 

http:/ / www.nwrac.org/whole-school/index.html 

Using Data to Improve Student Achievement Offers several tools to analyze and interpret data that were designed to aid schools in ana- 
lyzing Kentucky school data, but provide helpful examples for all school personnel, 
http:/ /www.kde.state.ky.us/oapd/rsc6/dataanalysis.asp 

Disaggregation without Aggravation: Is a multimedia training package for sale from the Southwest Educational Development Laboratory 
that includes case studies of schools as well as training and facilitation materials. 
http://www.sedl.org/pubs/catalog/items/teaching06.html 

Using Data to Improve Schools: 

http://www.aasa.org/cas/UsingDataToImproveSchools.pdf 

American Association of School Administrators' Center for Accountability Solutions: http://www.aasa.org/cas/ 

Fundamental Assessment Principles for Teachers and School Administrators: Article, 
http: / /ericae.net/pare/getvn.asp?v=7&n=8 
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Data Disaggregation Tools 

Ease-e Data Analyzer by TetraData: Is an online tool for data warehousing, mining, analysis, and reporting. Data can be disaggre- 
gated and reported by gender, ethnicity, and socioeconomic status. Standards can be created, tracked and analyzed with graphs 
and reports generated. TetraData also offers a Data Manager tool to assist in the import and export of data from a variety of 
sources, http://www.ease-e.com/ease-e/products/products_services_analyzer.asp 

Goalview: Offers teachers and administrators the ability to set goal, track performance, and report results especially for special needs 
and Title I students. Includes Gk)alcard, an individual report of student progress on standards that can be disseminated to teachers and 
parents; and a database of 250,000 Education Standards and 10,000 Specif Education goals to assist in creating goals and lEPs. 
Provides instructional resources as well as access to Federal and state laws and regulations. 
http://goal view, school . aol .com/GOALVIEW/index . asp 

SchoolNet: Offers a number of web-based tools for data collection and analysis as well as aligning curriculum and instruction to 
standards, http://www.schoolnet.com 

Socrates, by the Center for Resource Management: Offers tools and services to help schools import data, analyze, and ask good ques- 
tions. Has a capacity of 2 billion pieces of information and allows unlimited import of any electronic data source into a relational data 
base that can disaggregate data for one-to-one, one-to-many, or many-to-many relationship analyses. 
http://www.crminc.com/socrates/default.htm 

Quality School Portfolio: Is a free data analysis system consisting of two tools: a Data Manager, which allows a school or district to 
import data, disaggregate the data, and report about the data; and a Resource Kit, which consists of 21 tools to collect data includ- 
ing survey instruments, observation protocols, and questionnaires. Tools can collect information in these areas: Safety & Security, 
Parent Involvement, Professional Development, Curriculum & Instruction, Technology & Innovation, and Special Programs. QSP 
can house 8,000 records with 120 variables for 7 years. Data analysis is limited to mean, median, count, percentage, and distribu- 
tions. Displays are limited to pie charts, tables, bar graphs, cross tabulation, line chart, and histogram. 
http://qsp.cse.ucla.edu/mainSub/whatSet.html 

Source: Slowinski, 2002. 



exposed, and to keep a cumulative record of the knowledge and 
skills students may or may not have acquired. According to 
CRM, the SOCRATES™ system has import and data merge capa- 
bilities that can be used to integrate data from school records, 
administrative files, and standardized assessments to answer a 
wide range of questions. 

Obviously, other programs exist and are being developed. 

For example, Edmund W. Gordon was involved in the conceptual- 
ization of programs developed several years ago for the Prince 
Georges County Maryland Public Schools by the College Board’s 
Equity 2000 initiative and the Montgomery County Maryland 
Public Schools. Their goal was to encourage teachers and princi- 
pals to collect and interpret student, staff, and program data by 
providing access to desktop computers programmed to process 
such data. While the technical demands were modest, the pro- 
gram enabled immediate access to both aggregated and disaggre- 
gated student and program data. Personnel were thus able to 
monitor the dynamics of student placements, evaluate program 
effectiveness, gauge individual and collective student progress, 
and examine the relationships between these variables. The diffi- 
culties in these efforts were related to (1) gaining access to the 
relevant data to enter into the systems, (2) designing and man- 
aging the data entry process, and (3) developing the human capi- 
tal necessary to support and utilize the systems. 

An interesting effort to apply data-driven decision making 
was evaluated by Madhabi Chatterji of Teachers College, 
Columbia University. It involved using district level databases in 
a needs assessment for the Safe and Drugs Free Schools (SDFS) 
program in two Florida school districts for the purpose of initiat- 
ing new programs. The effort was made in response to a state- 
sponsored competition for federal funds under the Goals 2000 
initiative at a mid-size Florida school district. A review of the lit- 
erature on student risk and resiliency factors by the School and 



Community Advisory Council enabled the identification of three 
categories of indicators that would define a ‘Tiigh need school” for 
the SDFS program. Three categories of data elements, extracted 
from the district’s management information system and the 
Florida Information and Resource Network System, included: (1) 
academic risk indicators (student achievement data), (2) demo- 
graphic risk indicators (SES, mobility, attendance); and (3) 
social-behavioral risk indicators (crime, violence, truancy, and 
substance abuse). Reports showing school level aggregates of 
individual data elements and their interrelationships were pre- 
pared for district level decision makers. Subsequently, stakehold- 
ers were led by the evaluator through a systematic set of steps to 
arrive at a ranked list of schools that had the highest needs pro- 
files for SDFS funding. 

While space constraints prevent an elaboration of all exist- 
ing relational data management programs, the following pro- 
grams/data support resources may be of interest: 

• The Quality School Portfolio (developed by the Center for 
Research on Evaluation, Standards and Student Testing 
(CRESST) is a web-based mechanism for disaggregating 
tests and other data and examining data as mandated by 
state accountability systems and by the No Child Left 
Behind Legislation; 

• EDexplore™, developed by Edsmart Inc., manages data 
processes from data extraction and transformation to loading 
and hosting the resulting database on its servers. Clients, 
who gain access via the internet, receive training and ongo- 
ing support on the use of data to inform decisions. 

• ACCOUNT allows selected personnel access to relevant infor- 
mation concerning related and disparate aspects of schooling, 
including trends in attendance, test scores, finances, and stu- 
dent demographics. Similar to EDexplore™, ACCOUNT also 
integrates separate databases into a data warehouse. 
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Additional data disaggregation and management tools have 
been identified by Slowinski (2002) and are described in the box 
on page three. While these resources are clearly useful, it is impor- 
tant to address the criteria to which relational data management 
systems should adhere. CRESST suggests that data management 
systems should not only utilize various types of data from diverse 
sources, but should also include relevant information concerning 
program, student, and staff characteristics to (1) provide contexts 
for interpreting student achievement; (2) account for student out- 
comes such as achievement, attendance, mobility, and rates of 
retention in grade, dropout, and graduation; and (3) provide data 
not only on the available instructional resources and curriculiun 
materials, but also on the degree to which students are provided 
with adequate opportunities to master content specified in state 
and other curriculum standards. 

Conclusion 

For years, policy-makers, researchers, and practitioners have 
depended on summary descriptions of aggregated data, or on 



rather primitive analyses of the relationships between static cat- 
egories of these data as the basis for education related decisions. 
Too often we have had to resort to variably informed estimates 
as the basis for decision making — an approach to data manage- 
ment with serious limitations. The development of relational 
program and student data management systems provides 
schools increased capacity to document and understand the char- 
acteristics of respective inputs and outcomes by categories of stu- 
dents, staff, and programs in relation to differing educational 
contexts and circumstances. Effective data management systems 
can make this capacity readily available in “user fnendly*’ for- 
mats to people at various levels in decision making. Clearly, solv- 
ing the recalcitrant problem of reducing differences in the acade- 
mic achievement of diverse learners can be facilitated by the 
capacity to use relevant data to inform educational decisions. 
This renders use of relational program and student data man- 
agement systems an important component of an informed strate- 
gy for closing the academic achievement gap. 

— Edmund W, Gordon and Beatrice L, Bridglall 
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